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ABSTRACT 

A  person’s  behavior  provides  significant  information  about  their 
emotional  state,  attitudes,  and  attention.  Our  goal  is  to  create  vir¬ 
tual  humans  that  convey  such  information  to  people  while  interact¬ 
ing  with  them  in  virtual  worlds.  The  virtual  humans  must  respond 
dynamically  to  the  events  surrounding  them,  which  are  fundamen¬ 
tally  influenced  by  users’  actions,  while  providing  an  illusion  of 
human-like  behavior.  A  user  must  be  able  to  interpret  the  dynamic 
cognitive  and  emotional  state  of  the  virtual  humans  using  the  same 
nonverbal  cues  that  people  use  to  understand  one  another.  Towards 
these  goals,  we  are  integrating  and  extending  components  from 
three  prior  systems:  a  virtual  human  architecture  with  a  range  of 
cognitive  and  motor  capabilities,  a  model  of  emotional  appraisal, 
and  a  model  of  the  impact  of  emotional  state  on  physical  behavior. 
We  describe  the  key  research  issues,  our  approach,  and  an  initial 
implementation  in  an  Army  peacekeeping  scenario. 

1.  INTRODUCTION 

A  person’s  emotional  state  influences  them  in  many  ways.  It  im¬ 
pacts  their  decision  making,  actions,  memory,  attention,  voluntary 
muscles,  etc.,  all  of  which  may  subsequently  impact  their  emo¬ 
tional  state  (e.g.,  see  [4]).  This  pervasive  impact  is  reflected  in 
the  fact  that  a  person  will  exhibit  a  wide  variety  of  nonverbal  be¬ 
haviors  consistent  with  their  emotional  state,  behaviors  that  can 
serve  a  variety  of  functions  both  for  the  person  exhibiting  them 
as  well  as  for  people  observing  them.  For  example,  shaking  a  fist 
at  someone  plays  an  intended  role  in  communicating  information. 
On  the  other  hand,  behaviors  such  as  rubbing  one's  thigh,  averting 
gaze,  or  a  facial  expression  of  fear  may  have  no  explicitly  intended 
role  in  communication.  Nevertheless,  these  actions  may  suggest 
considerable  information  about  a  person’s  emotional  arousal,  their 
attitudes,  and  their  focus  of  attention. 

Our  goal  is  to  create  virtual  humans  that  convey  these  types  of  in¬ 
formation  to  humans  while  interacting  with  them  in  virtual  worlds. 
We  are  interested  in  virtual  worlds  that  offer  human  users  an  en¬ 


gaging  scenario  through  which  they  will  gain  valuable  experience. 
For  example,  a  young  Army  lieutenant  could  be  trained  for  a  peace¬ 
keeping  mission  by  putting  him  in  virtual  Bosnia  and  presenting 
him  with  the  sorts  of  situations  and  dilemmas  he  is  likely  to  face. 
In  such  scenarios,  virtual  humans  can  play  a  variety  of  roles,  such 
as  an  experienced  sergeant  serving  as  a  mentor,  soldiers  serving 
as  his  teammates,  and  the  local  populace.  Unless  the  lieutenant  is 
truly  drawn  into  the  scenario,  his  actions  are  unlikely  to  reflect  the 
decisions  he  will  make  under  stress  in  real  life.  The  effectiveness 
of  the  training  depends  on  our  success  in  creating  engaging,  be¬ 
lievable  characters  that  convey  a  rich  inner  dynamics  that  unfolds 
in  response  to  the  scenario. 

Thus,  our  design  of  the  virtual  humans  must  satisfy  three  require¬ 
ments.  First,  they  must  be  believable;  that  is,  they  must  provide 
a  sufficient  illusion  of  human-like  behavior  that  the  human  user 
will  be  drawn  into  the  scenario.  Second,  they  must  be  respon¬ 
sive;  that  is,  they  must  respond  to  the  events  surrounding  them, 
which  will  be  fundamentally  influenced  by  the  user’s  actions.  Fi¬ 
nally,  they  must  be  interpretable;  the  user  must  be  able  to  interpret 
their  response  to  situations,  including  their  dynamic  cognitive  and 
emotional  state,  using  the  same  nonverbal  cues  that  people  use 
to  understand  one  another.  Thus,  our  virtual  humans  cannot  sim¬ 
ply  create  an  illusion  of  life  through  cleverly  designed  randomness 
in  their  behavior;  their  inner  behavior  must  respond  appropriately 
to  a  dynamically  unfolding  scenario,  and  their  outward  behavior 
must  convey  that  inner  behavior  accurately  and  clearly. 

This  paper  describes  our  progress  towards  a  model  of  the  outward 
manifestations  of  an  agent’s  emotional  state.  Our  work  integrates 
three  previously  implemented  systems.  The  first,  Steve  [18,  20, 
19],  provides  an  architecture  for  virtual  humans  in  3D  simulated 
worlds  that  can  interact  with  human  users  as  mentors  and  team¬ 
mates.  Although  Steve  did  not  include  any  emotions,  his  broad 
capabilities  provide  a  foundation  for  the  virtual  humans  towards 
which  we  are  working.  The  second,  Emile  [8],  focuses  on  emo¬ 
tional  appraisal:  how  emotions  arise  from  the  relationship  between 
environmental  events  and  an  agent's  plans  and  goals.  The  third, 
IPD  [11],  contributes  a  complementary  model  of  emotional  ap¬ 
praisal  as  well  as  a  model  of  the  impact  of  emotional  state  on 
physical  behavior.  The  integration  of  these  three  systems  provides 
an  initial  model  of  virtual  humans  for  experiential  learning  in  en¬ 
gaging  virtual  worlds.  Our  work  is  part  of  a  larger  effort  to  add  a 
variety  of  new  capabilities  to  Steve,  including  more  sophisticated 
support  for  spoken  dialog  and  a  more  human-like  model  of  per¬ 
ception  [17]. 
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Figure  1:  An  interactive  peacekeeping  scenario  featuring  (left  to  right)  a  sergeant,  a  mother,  and  a  medic 


2.  EXAMPLE  SCENARIO 

To  illustrate  our  vision  for  virtual  humans  that  can  interact  with 
people  in  virtual  worlds,  we  have  implemented  an  Army  peace¬ 
keeping  scenario,  which  has  been  viewed  by  several  hundred  peo¬ 
ple  and  was  favorably  viewed  by  many  domain  experts  [21].  As 
the  simulation  begins,  a  human  user,  playing  the  role  of  a  U.S. 
Army  lieutenant,  finds  himself  in  the  passenger  seat  of  a  simu¬ 
lated  vehicle  speeding  towards  a  Bosnian  village  to  help  a  platoon 
in  trouble.  Suddenly,  he  rounds  a  corner  to  find  that  one  of  his  pla¬ 
toon’s  vehicles  has  crashed  into  a  civilian  vehicle,  injuring  a  local 
boy  (Figure  1).  The  boy’s  mother  and  an  Army  medic  are  hunched 
over  him,  and  a  sergeant  approaches  the  lieutenant  to  brief  him  on 
the  situation.  Urgent  radio  calls  from  the  platoon  downtown,  as 
well  as  occasional  explosions  and  weapons  fire  from  that  direction, 
suggest  that  the  lieutenant  send  his  troops  to  help  them.  Emotional 
pleas  from  the  boy’s  mother,  as  well  as  a  grim  assessment  by  the 
medic  that  the  boy  needs  a  medevac  immediately,  suggest  that  the 
lieutenant  instead  use  his  troops  to  secure  a  landing  zone  for  the 
medevac  helicopter.  The  lieutenant  carries  on  a  dialog  with  the 
sergeant  and  medic  to  assess  the  situation,  issue  orders  (which  are 
carried  out  by  the  sergeant  through  four  squads  of  soldiers),  and 
ask  for  suggestions.  His  decisions  influence  the  way  the  situation 
unfolds,  culminating  in  a  glowing  news  story  praising  his  actions 
or  a  scathing  news  story  exposing  the  flaws  in  his  decisions  and 
describing  their  sad  consequences. 

While  the  current  implementation  of  the  scenario  offers  little  flex¬ 
ibility  to  the  lieutenant,  it  provides  a  rich  test  bed  for  our  research. 
Currently,  the  mother,  medic  and  sergeant  are  implemented  as 
Steve  agents.  All  other  virtual  humans  (a  crowd  of  locals  and  four 
squads  of  soldiers)  are  scripted  characters.  We  will  illustrate  ele¬ 
ments  of  our  design  using  the  mother,  since  her  emotions  are  both 
the  most  dramatic  in  the  scenario  and  the  most  crucial  for  influ¬ 
encing  the  lieutenant’s  decisions  and  emotional  state.  While  our 
current  implementation  of  the  mother  includes  a  preliminary  inte¬ 
gration  of  Steve,  Emile,  and  IPD,  the  focus  of  this  paper  is  on  a 
more  general  integration  that  goes  beyond  that  implementation. 

3.  RELATED  WORK 

Several  researchers  have  considered  computational  models  of  emo¬ 
tion  to  drive  the  behavior  of  synthetic  characters.  These  approaches 
can  be  roughly  characterized  as  being  either  communication  driven 


or  appraisal  driven.  Communication  driven  means  that  one  se¬ 
lects  a  communication  act  and  then  chooses  emotional  expression 
based  on  some  desired  impact  it  will  have  on  the  user.  For  ex¬ 
ample,  Cosmo  [10]  makes  a  one-to-one  mapping  between  speech 
acts  and  emotional  behaviors  that  reinforce  the  act  (congratulatory 
acts  trigger  an  admiration  emotive  intent  that  is  conveyed  with  ap¬ 
plause).  Ball  and  Breese  [2]  intend  to  convey  a  sense  of  empathy 
by  expressing  behaviors  that  mirror  the  assessed  emotional  state  of 
the  user.  Poggi  and  Pelachaud  [16]  use  facial  expressions  to  con¬ 
vey  the  performative  of  a  communication  act,  showing  “potential 
anger’’  to  communicate  that  the  agent  will  be  angry  if  a  request  is 
not  fulfilled. 

In  contrast,  appraisal  theories  focus  on  the  apparent  evaluative 
function  emotions  play  in  human  reasoning.  Appraisal  theories 
view  emotion  as  arising  from  some  assessment  of  an  agent’s  in¬ 
ternal  state  vis-a-vis  its  external  environment  (e.g.,  is  this  event 
contrary  to  my  desire?).  Such  appraisals  can  be  used  to  guide 
decision-making  and  behavior  selection.  For  example,  Beaudoin 
[3]  uses  them  to  set  goal  priorities  and  guide  meta-planning.  They 
can  also  serve  as  a  basis  for  communicating  information  about  the 
agent's  assessment,  though  not  in  the  intentional  way  viewed  by 
communicative  models.  Appraisal  methods  are  useful  for  giving 
coherence  to  an  agent’s  emotional  dynamics  that  can  be  lacking 
in  purely  communicative  models.  This  coherence  is  essential  for 
conveying  a  sense  of  believability  and  realism  [13], 

These  approaches  are  complementary,  though  few  approaches  have 
considered  how  to  combine  them  into  a  coherent  approach.  An 
exception  is  IPD  [11],  which  represents  both  appraised  emotional 
state  and  communicative  intent  and  selectively  expresses  one  or 
the  other  based  on  a  simple  threshold  model.  In  the  discussion 
that  follows  we  will  focus  on  the  problem  of  emotional  appraisal. 

4.  MODELING  EMOTIONS 
4.1  Emotional  State 

Our  primary  design  goal  is  to  create  general  mechanisms  that  are 
not  tied  to  the  details  of  any  specific  domain.  Thus,  emotional  ap¬ 
praisals  do  not  refer  to  features  of  the  environment  directly,  but 
reference  abstract  features  of  the  agent’s  information  processing. 
Appraisals  and  re-appraisals  are  triggered  reflexively  by  changes 


in  an  agent's  mental  representations,  which  are  in  turn  changed 
by  general  reasoning  mechanisms.  External  events  can  indirectly 
impact  these  internal  representations.  For  example,  the  agent  may 
perceive  an  event  (e.g.,  soldiers  are  leaving  the  accident  scene), 
and  then  a  general  reasoning  mechanism  (e.g.  Steve’s  task  reason¬ 
ing)  infers  the  consequences  of  this  event  (e.g.,  the  departure  of 
troops  violates  the  precondition  of  them  treating  my  child).  Alter¬ 
natively,  these  representations  may  change  as  the  result  of  inter¬ 
nal  decision-making  (e.g.,  forming  an  intention  to  violate  a  social 
obligation).  Appraisal  frames  key  off  of  features  of  these  repre¬ 
sentations  (e.g.,  the  negative  impact  of  a  precondition  violation  on 
goal  probability  or  the  violation  of  the  social  norm  of  meeting  your 
obligations).  The  dynamics  of  behavior  and  emotional  expression 
is  thereby  driven  by  the  mechanisms  that  update  an  agent’s  mental 
representations. 

The  Emile  system  [8]  illustrates  this  approach  by  using  a  domain- 
independent  planning  approach  to  represent  mental  state  and  make 
inferences,  but  Emile  is  therefore  largely  restricted  to  appraisals 
related  to  an  agent’s  goals  and  changes  to  the  probability  of  goal 
attainment.  IPD  [11]  explores  a  much  richer  notion  of  appraisal 
including  a  variety  of  emotional  coping  strategies  and  a  richer  no¬ 
tion  of  social  relationships  and  ego  identity,  but  is  implemented  in 
terms  of  a  less  general  inference  mechanism.  In  our  joint  work, 
we  begin  to  bridge  the  gap  between  by  buttressing  Emile-style  ap¬ 
praisals  with  a  broader  repertoire  of  inference  strategies.  Initially, 
we  are  augmenting  plan-based  appraisals  with  those  related  to  di¬ 
alog,  incorporating  methods  for  representing  and  updating  the  dis¬ 
course  state  and  social  obligations  that  arise  from  discourse  [15]. 
These  structures  can  then  serve  as  the  basis  for  a  more  uniform  rep¬ 
resentation  of  social  commitments  and  obligations  beyond  those 
strictly  related  to  dialog,  and  that  interact  with  plan-based  reason¬ 
ing  (e.g.  one  is  obligated  to  redress  wrongs).  This  will  allow  the 
system  to  model  some  of  the  rich  dynamics  of  emotions  relating 
to  discourse  that  IPD  supports,  but  do  so  in  a  more  structured  way. 

Emile  contains  a  set  of  recognition  rules  that  scan  an  agent’s  in¬ 
ternal  representations  and  generate  an  appraisal  frame  whenever 
certain  features  are  recognized.  For  example,  whenever  the  agent 
adopts  a  new  goal  (or  is  informed  of  a  goal  of  some  other  agent), 
frames  are  created  to  track  the  status  of  that  goal.  Each  appraisal 
frame  describes  the  appraised  situation  in  terms  of  a  number  of 
specific  features  proposed  by  the  OCC  model  [14].  These  include 
the  point  of  view  from  which  the  appraisal  is  formed,  the  desir¬ 
ability  of  the  situation,  whether  the  situation  has  come  to  pass  or  is 
only  a  possibility,  whether  the  situation  merits  praise  or  blame  (we 
do  not  model  the  appealingness  of  domain  objects,  an  important 
factor  in  OCC).  For  example,  if  someone  threatens  my  goal,  from 
my  perspective  this  is  an  undesirable  possibility  that  merits  blame. 
Based  on  the  setting  of  the  various  features,  one  or  more  “emotion 
instances”  are  generated  according  to  OCC.  Emile  incorporates 
a  decision-theoretic  method  for  assigning  intensities  to  these  in¬ 
stances.  Intensity  values  decay  over  time  but  are  “re-energized” 
whenever  the  underlying  representational  structures  are  manipu¬ 
lated. 

Individual  emotion  instances  are  aggregated  into  “buckets”  corre¬ 
sponding  to  emotions  of  the  same  type.  Thus,  threats  to  multiple 
goals  will  be  aggregated  into  an  overall  level  of  fear.  In  this  sense, 
Emile  contains  a  two-level  representation  of  emotional  state.  Emo¬ 
tion  instances  are  the  level  at  which  some  semantic  connection  is 
still  retained  between  the  emotion  and  the  mental  structures  un¬ 
derlying  the  appraisal.  Thus,  the  agent  has  the  capability  of  an¬ 
swering  questions  about  its  emotion  state  (e.g.,  “Why  are  you  so 
angry?”).  The  aggregate  buckets  roughly  correspond  to  the  over¬ 


all  assessment  of  the  agent’s  emotional  state  and  are  used  to  drive 
the  physical  focus  modes  discussed  below.  This  model  does  yet 
support  some  of  the  subtle  dynamics  displayed  by  IPD  which  can 
convey  an  overall  tone  yet  make  agile  transitions  between  con¬ 
veyed  emotional  state.  We  are  exploring  some  different  strategies 
for  achieving  this  effect,  including  aggregating  subsets  of  appraisal 
frames  on  demand  depending  of  the  current  discourse  state,  or  by 
adding  another  aggregate  layer  that  incorporates  a  different  decay 
rate  into  appraisal  intensities. 

4.2  The  Effect  of  Emotions  on  Behavior 

The  agents  we  design  incorporate  a  wide  range  of  outward  behav¬ 
iors  in  order  to  interact  believably  with  the  environment  as  well 
as  other  agents  and  humans.  Their  bodies  have  fully  articulated 
limbs,  facial  expressions,  and  sensory  apparatus.  They  can  move 
in  the  environment,  manipulate  objects  and  direct  their  gaze  in  ap¬ 
propriate  ways.  They  are  capable  of  rich,  multi-modal  communi¬ 
cation  that  incorporates  both  verbal  behaviors  as  well  as  nonverbal 
behaviors.  In  addition,  they  have  facial  expressions,  body  postures 
and  the  ability  to  perform  various  kinds  of  gestures.  The  key  chal¬ 
lenge  for  the  agent  design  is  to  manage  this  flexibility  in  the  agent’s 
physical  presence  in  a  way  that  conveys  consistent  emotional  state 
and  individual  differences.  To  address  these  challenges,  we  rely  on 
a  wide  range  of  work  in  human  emotion,  social  behavior,  clinical 
psychology,  animation  and  the  performance  arts. 

For  example,  psychological  research  on  emotion  reveals  its  perva¬ 
sive  impact  on  physical  behavior  such  as  facial  expressions,  gaze 
and  gestures  [1,  5,  6],  These  behaviors  suggest  considerable  infor¬ 
mation  about  emotional  arousal,  attitudes  and  attention.  For  ex¬ 
ample,  depressed  individuals  may  avert  their  gaze  and  direct  their 
gestures  inward  towards  their  bodies  in  self-touching  behaviors. 

Note  that  such  movements  also  serve  to  mediate  the  information 
available  to  the  individual.  For  example,  if  a  depressed  individ¬ 
ual’s  head  is  lowered,  this  also  regulates  the  information  available 
to  the  individual.  Orienting  on  an  object  of  fear  or  anger  brings 
the  object  to  the  focus  of  perceptual  mechanisms,  which  may  have 
indirect  influences  on  cognition  and  cognitive  appraisal  by  influ¬ 
encing  the  content  of  working  memory.  Even  a  soothing  behavior 
like  rubbing  an  arm  may  serve  to  manage  what  a  person  attends  to 
[7], 

These  findings  provide  a  wealth  of  data  to  inform  agent  design 
but  often  leave  open  many  details  as  to  how  alternative  behaviors 
are  mediated.  The  agent  technology  we  use  allows  one  to  create 
rich  physical  bodies  for  intelligent  characters  with  many  degrees 
of  physical  movement.  This  forces  one  to  directly  confront  the 
problem  of  emotional  consistency.  For  example,  an  “emotionally 
depressed”  agent  might  avert  gaze,  be  inattentive,  perhaps  hug¬ 
ging  themselves.  However,  if  in  subsequent  dialog  the  agent  used 
strong  communicative  gestures  such  as  beats  [12],  then  the  behav¬ 
ior  might  not  “read”  correctly.  This  issue  is  well  known  among 
animators.  For  example,  Frank  Thomas  and  Ollie  Johnston,  two 
of  Disney’s  “grand  old  men,”  observe  that  “the  expression  must 
be  captured  throughout  the  whole  body  as  well  as  in  the  face”  and 
“any  expression  will  be  weakened  greatly  if  it  is  limited  only  to 
the  face,  and  it  can  be  completely  nullified  if  the  body  or  shoulder 
attitude  is  in  any  way  contradictory”  [22]. 

Implicit  in  these  various  concerns  is  that  the  agent  has  what  amounts 
to  a  resource  allocation  problem.  The  agent  has  limited  physi¬ 
cal  assets,  e.g.,  two  hands,  one  body,  etc.  At  any  point  in  time, 
the  agent  must  allocate  these  assets  according  to  a  variety  of  de- 


mands,  such  as  performing  a  task,  communicating,  or  emotionally 
soothing  themselves.  For  instance,  the  agent's  dialog  may  be  sug¬ 
gestive  of  a  specific  gesture  for  the  agent's  arms  and  hands  while 
the  emotional  state  is  suggestive  of  another.  The  agent  must  medi¬ 
ate  between  these  alternative  demands  in  a  fashion  consistent  with 
their  goals  and  their  emotional  state.  From  the  human  participant’s 
perspective,  we  expect  this  consistency  in  the  agent’s  behavior  to 
support  believability  and  interpretability. 

To  address  this  problem  we  rely  on  the  Physical  Focus  model  [11], 
a  computational  technique  inspired  by  work  on  nonverbal  behavior 
in  clinical  settings  [7]  and  Lazarus’s  [9]  delineation  of  emotion- 
directed  versus  problem-directed  coping  strategies.  The  Physical 
Focus  model  bases  an  agent’s  physical  behavior  in  terms  of  what 
the  character  attends  to  and  how  they  relate  to  themselves  and 
the  world  around  them,  specifically  whether  they  are  focusing  on 
themselves  and  thereby  withdrawing  from  the  world  or  whether 
they  are  focusing  on  the  world,  engaging  it.  In  other  words,  the 
model  bases  physical  behavior  on  how  the  agent  is  choosing  to 
cope  with  the  world. 

The  model  organizes  possible  behaviors  around  a  set  of  modes. 
Behaviors  can  be  initiated  via  requests  from  the  planner/executor 
or  started  spontaneously  when  the  body  is  not  otherwise  engaged. 
At  any  point  in  time,  the  agent  will  be  in  a  unique  mode  based  on 
the  current  emotional  state. 

In  the  current  work,  we  model  three  modes  of  physical  focus: 
Body  Focus,  Transitional  Focus  and  Communicative  Focus  (as  op¬ 
posed  to  the  five  modes  discussed  in  [11]  and  identified  in  Freed¬ 
man’s  work).  The  focus  mode  impacts  behavior  along  several  key 
dimensions,  including  which  behaviors  are  available,  how  behav¬ 
iors  are  performed,  how  attentive  to  external  events  the  agent  is, 
and  which  actions  the  agent  selects  and  how  it  speaks. 

The  Body  Focus  mode  models  a  self-focused  attention,  away  from 
the  world,  and  is  also  designed  to  reveal  considerable  depression 
or  guilt.  The  prevalent  type  of  gestures  available  to  the  agent  in 
this  mode  are  self-touching  gestures,  or  adaptors  [5],  which  in¬ 
volve  hand-to-body  gestures  that  appear  soothing  (e.g.,  rhythmic 
stroking  of  forearm)  or  self-punitive  (e.g.,  squeezing  or  scratching 
of  forearm).  Conversely,  communicative  gestures  such  as  deic¬ 
tic  or  beat  gestures  [12]  are  muted  in  this  mode  both  in  terms  of 
their  numbers  and  dynamic  extent.  Indeed,  overall  gesturing  in 
this  mode  is  muted  in  its  dynamics.  Also,  the  agent  is  less  atten¬ 
tive  to  the  environment  and  in  particular  exhibits  considerable  gaze 
aversion  and  downward  looking  gaze.  The  agent's  verbal  behavior 
is  inhibited  and  marked  by  pauses.  In  terms  of  action  selection, 
the  agent  has  a  reduced  preference  for  communicative  acts. 

Transitional  Focus  indicates  a  less  divided  attention,  less  depres¬ 
sion  or  guilt,  a  burgeoning  willingness  to  take  part  in  the  conversa¬ 
tion,  milder  conflicts  with  the  problem  solving  and  a  closer  relation 
to  the  listener.  This  mode's  gesturing  is  marked  by  hand-to-hand 
gestures  (such  as  rubbing  hands  or  hand  fidgetiness)  and  hand- 
to-object  gestures,  such  as  playing  with  a  pen.  There  are  more 
communicative  gestures  in  this  mode  but  they  are  still  muted  or 
stilted. 

Finally,  Communicative  Focus  is  designed  to  indicate  a  full  will¬ 
ingness  to  engage  the  world  in  dialog  and  action.  In  this  mode, 
the  agent’s  full  range  of  communicative  gestures  are  available.  On 
the  other  hand,  adaptors  are  muted  and  occur  only  when  the  agent 
is  listening  or  otherwise  not  occupied.  In  contrast,  the  commu¬ 
nicative  gestures  are  more  exaggerated,  both  in  terms  of  numbers 


and  physical  dynamics.  The  agent  is  also  more  attentive  to  and 
responsive  to  events  in  the  environment. 

Transition  between  modes  is  currently  based  on  emotional  state 
derived  from  the  appraisal  model.  High  levels  of  sadness,  de¬ 
creased  hope,  or  guilt,  in  absolute  terms  and  relative  to  other  emo¬ 
tional  buckets,  induces  transitions  away  from  Communicative  Fo¬ 
cus  and  towards  Body  Focus.  Increased  hope  or  anger  induces 
transitions  towards  Communicative  Focus.  The  transitions  are  de¬ 
signed  to  be  “sticky”  so  that  the  agent  does  not  readily  pop  into 
and  then  out  of  a  mode.  Transitional  Focus,  true  to  its  name,  lies 
between  the  other  two  modes.  However,  extreme  emotional  state 
can  bypass  it. 

Grouping  behaviors  into  modes  attempts  to  (a)  mediate  competing 
demands  on  an  agent’s  physical  resources  and  (b)  coalesce  behav¬ 
iors  into  groups  that  provide  a  consistent  interpretation  about  how 
the  agent  is  relating  to  its  environment.  It  is  designed  to  do  this  in 
a  fashion  consistent  with  emotional  state,  with  the  intent  that  it  be 
general  across  agents.  However,  realism  also  requires  that  specific 
behaviors  within  each  mode  incoiporate  individual  differences,  as 
in  human  behavior.  For  example,  we  would  not  expect  a  mother's 
repertoire  of  gestures  to  be  identical  to  that  of  an  army  sergeant. 
Indeed,  the  current  implementation  is  a  step  en  route  to  a  fuller 
rendering  of  a  coping  model:  a  model  of  how  the  agent  chooses  to 
cope  with  emotional  stress  that  includes  both  emotional  appraisal 
as  well  as  individual  differences  (personality). 

5.  CONCLUSION 

The  challenge  of  this  research  is  not  an  easy  one.  We  seek  to 
create  virtual  humans  whose  behavior  is  a  believable  rendition  of 
human-like  behavior,  is  responsive  to  dynamically  unfolding  sce¬ 
narios  and  is  interpretable  using  the  same  cues  that  people  use  to 
understand  each  other.  A  key  factor  for  success  will  be  a  faithful 
rendering  of  the  cause  of  emotion  and  its  outward  manifestation. 
To  address  that  challenge,  we  have  begun  to  integrate  components 
from  three  previous  systems:  Steve’s  virtual  human  architecture, 
Emile’s  emotional  appraisal  and  IPD’s  complementary  model  of 
emotional  appraisal  and  its  impact  on  physical  behavior. 

An  initial  integration  has  already  been  realized  and  implemented 
in  the  mother  character  of  our  Army  peacekeeping  scenario.  How¬ 
ever,  our  integration  efforts  are  ongoing.  Our  current  efforts  are 
on  generalizing  that  integration  and  incorporating  more  of  the  IPD 
social-based  appraisals  into  the  general  inferencing  mechanisms 
in  Steve  and  Emile. 
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